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The results, and their implications, of research 
described in the report "Education Information: Changes in Funds and 
Priorities Have Affected Production and Quality" are discussed* 
Prepared by the General Accounting Office (GAO), the above research 
report covered selected years between 1973 and 1986. High-quality 
information on education is essential to the tasks of overseeing the 
use of federal resources, assessing the nation's progress, and 
identifying areas for improvement. In the last decade, federally 
sponsored research and statistical and evaluative information have 
declined dramatically. As J ass information was produced, there was 
also a shift from new data production to service-oriented ac+ivities. 
Three statistical series were evaluated: (1) the National cessment 
of Educational Progress (NAEP); (2) the Common Core of Data (CCD); 
and (3) the Fast Response Survey System (FRSS). The NAEP was rated 
high for relevance, timeliness, adequacy, and impact. The CCD was not 
rated highly, and the FRSS received mixed reviews. The overall 
picture is one of many reasons for concern about the production and 
quality of educational information, due mainly to the following 
interacting factors: (1) declines in funding; (2) the role of 
congressional mandates; and (3) changes in leadership and priorities. 
It is noted that the Department of Education does not share this 
pessimistic assessment made by the GAO. (SLD) 
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MR. CHAIRMAN AND MEMBERS OF THE COMMITTEE: 

It is a pleasure to be here today to report to you on our study of 
the condition of information on education in the United States. 
The results I present here come from our report entitled Education 
Information; Cnanges in Funds and Priorities Have Affected 
Productio n and Quality (GAO/PEMD-88-4 ) . Today, I would like to 
highlight our central findings and discuss their implications. Our 
work covered selected years between 1973 and 198 6. 

I want, in particular, to make three points. The first is that in 
my opinion, sound information about education is important — even 
vital--to educational reform and to oversight. The second point is 
that we have found reason for concern about the production of 
education data — that is, whether this sound information will be 
available. The third point is that the reasons for the problems we 
found are complex. They include lack of resources, but that is not 
the whole story. Thus, turning the situation around is likely to 
take time and to require in itself considerable information and 
monitoring . 
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The Importance of Information About Education 

As you know, although education in this country is a responsibility 
of the states, the federal government spent nearly $20 billion in 
fiscal year 1987 to support all levels of education. In 186 7, the 
Congress authorized the creation of a noncabinet Department of 
Education to obtain information on the condition of education for 
purposes of identifying emerging needs, determining how well 
programs are working, and promoting educational improvement, an 
authorization continued and expanded over the years. 1 I think that 
today, no less than in 1867, it is critical to have high-quality 
informa "on for overseeing federal educational resources, assessing 
the progress the nation has made in improving educational access 
and quality, and identifying shortfalls yet to be dealt with 
successfully. By high-quality information, I mean information that 
is relevant, timely, technically adequate, and usable for policy 
decisions . 



^-Although the Department of Education was not made a cabinet 
department until 197 9, we refer to it as the Department of 
Education. 
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Reason for Concern; The Production of Information 



Federally sponsored research and statistical and evaluative 
information on education declined dramatically during the past 
decade. With regard to research, the number of grants and 
contracts awarded decreased from 476 in 1980 to 168 in 1985. The 
change from 1980 to 1985 was the most dramatic for the unsolicited 
proposal program in the National Institute of Education (NIE) 
because it was cut completely. 

Declines in awards for evaluations were also substantial. The high 
level of activity that began late in the 1970' s (80 or more awards 
annually) peaked at 119 in 1980 and began to drop in 1981, leveling 
off to between 25 and 28 activities annually. From 1980 to 1985, 
tne decline was 79 percent. The biggest overall decrease followed 
the passage of block grant legislation, vrtiich affected many of the 
elementary and secondary education programs that had previously 
received the bulk of the evaluation support and review. 

The total number of statistical surveys, planned or conducted by 
the National Center for Educational Statistics (NCES), now named 
the Center for Education Statistics (CES), grew by 49 percent (from 
37 to 55) between 1974 and 1980.2 However, between 1980 and 1983, 

2 In the department's October 1985 reorganization, NIE and NCES 
were discontinued as separate agencies and all their functions and 
activities were assigned to the five operating units of the Office 
of Educational Research and Improvement. (The five units are 
Office of Research, Center for Education Statistics, Programs for 
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the number of surveys fell by 31 percent, returning to its 1974 
aggregate level. In addition, the intervals between statistical 
data collections increased, and technical support to states for 
data collection was sharply reduced. For example, over the 9 years 
we examined, there was a 20-percent decline in annual surveys 
(which permit detailed analyses of trends) and an 83-percent 
increase in occasional or one-time surveys. 

Concern over the quantity of awards is heightened by three other 
significant changes. 

Shift Away From New Data Collection 

First, not only was less information produced: we also found 
changes in priorities. For the National Institute of Education's 
portfolio of activities, there was a shift away from new data 
production to service-oriented activities, such as dissemination of 
results and the provision of expert witnesses in civil rights 
cases. Sixty-five percent of NIE's 1980 awards were for new data 
collection, but only 11 percent of the 1985 awards were dedicated 
to this function. In our view, this shift was so dramatic that the 

the Improvement of Practice, Information Services, and Library 
Programs.) CES performs most of the former responsibilities of 
NCES. And although some NIE responsibilities have been transferred 
to the new units, the Office of Research now carries out the 
activities of NIE that we discuss in this statement. Because our 
review covers the period prior to the departmental reorganization, 
we refer to each unit by the name applicable during that period— 
that is, NIE, NCES, and the Office of Planning, Budget, and 
Evaluation (OPBE) . 



9 

ERIC 



6 



availability of up-to-date information to disseminate to teachers 
and other practitioners may be seriously jeopardized. 

Fewer Areas Investigated 

Second, fewer educational areas were investigated in 1985 than in 
1980 through research grants. In 1980, for example, 56 of 293 
awards for new data collection went toward studies of special 
populations, such as minorities and women. In 1985, there were 
five such studies. Some areas such as learning in nonschool 
settings and areas identified as "school problems" (including such 
issues as dropouts and delinquency) received no new data collection 
funds in 1985; in 1980, there were 33 awards. Even for the topics 
that have frequently been identified as important areas for 
educational improvement — for example, improving teacher 
preparation; strengthening curricula in mathematics, science, and 
English; more-effective instruction; classroom management and 
school leadership—there were few awards for new data collection in 
1985. 

Information Producers Changed 

Third, there was a shift among those who carried out the work of 
producing information, and the procurement process became more 
constrained. The proportion of research awards made to 
department-sponsored institutions (for example, laboratories and 



f 



9 

ERIC 



7 



national centers) increased substantially from 1980 to 1985. in 
1980, those institutions received 25 percent of the awards in three 
major program areas, compared to 56 percent in 1985. The 
cumulative result of various shifts in awards is that the majority 
of the department's information producers were ii.stitut ions or 
contractors. This shift is a concern because, while contracts may 
be the most applicable when there is a specific request for 
information (for example, a congressionally mandated study) or when 
continuity in data gathering is necessary (for example, in a 
statistical series), their use as the predominant vehicle for 
funding research may be inappropriate. Compared to the grants 
mechanism, for example, a contract is likely to constrain rather 
than broaden inquiry. 

Reason for Concern; The Quality of Information 

Turning to our concern for the soundness or quality of information, 
we reviewed in-depth evidence regarding four dimensions of 
quality— relevance, timeliness, technical adequacy, and impact — for 
three statistical series — the National Assessment of Educational 
Progress (NAEP), the Common Core of Data (CCD) for elementary and 
secondary education, and the Fast Response Survey System. 

In general, NAEP ranked high on all four dimensions, but it has 
suffered some decline in relevance and timeliness in adapting to 
fiscal constraints. In particular, it has been affected by 
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budgetary declines in two ways. The number of target populations 
was reduced from five principal groups (9-, 13-, and 17-year-olds 
in and out of school and adults) to three (9-, 13-, and 17-year- 
olds who remain in school), and the assessment cycle was altered 
from annual to biennial, or longer, some content areas only being 
assessed at 4- and 6-year intervals. This is important because the 
ability of NAEP to record changes in performance depends on 
maintaining short intervals between assessments. As an interval 
increases, the ability to signal changes becomes more limited. 
Further, many groups, including students younger than 9 years old, 
are not assessed by NAEP. 

The Common Core of Data— a primary source of nationwide information 
on elementary and secondary institutions— was not rated high on any 
of the four indicators, although >me data elements were found to 
be adequate. In general, data were not comparable across states, 
mainly because data elements were reported at different levels of 
aggregation or were derived from different definitions and data 
collection procedures. Problems with CCD have long been 
recognized, but few have been solved. Commendably, the department 
is currently making efforts to improve some parts of the CCD. 

The Fast Response Survey System was rated moderate to high on 
quality, especially given the low budgets associated with each 
survey. The system has been responsive to the information needs of 
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the requesters and has minimized time delays by releasing findings 
early. Reporting survey procedures could be improved, however. 
Methods for handling nonresponse and overall response rates were 
not always reported in sufficient detail to assess the quality of 
practice. 

Complex Influences on Production and Quality 

The overall picture, then, is one of many reasons for concern and 
also of some successes with regard to educational information, in 
examining the reasons for these problems and successes, we found 
that resources play a major role but that lack of money was not the 
only issue. 

The fiscal resources for the department increased in current 
dollars from approximately $6.1 billion in 1973 to $19.5 billion in 
1986— an increase of 220 percent, or 38 percent in 1972 dollars. 
The trends for fiscal support of the production of research and 
statistical and evaluative information were quite different. Since 
the mid-1970' s, NIE experienced a 79-percent reduction in constant 
1972 dollars; NCES experienced a 65-percent reduction; and the 
Office of Planning, Budget, and Evaluation's resources declined by 
64 percent. These reductions are in sharp contrast to the 38- 
percent increase in the federal investment in education over this 
same period. 
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Viewed another way, changes in fiscal resources for education 
information were more severe than was reposed for other federal 
agencies with similar missions. That is, while the real purchasing 
power of overall federal research funds grew by 3.7 percent between 
1980 and 1984, NIE's funding level declined by 48 percent. 

NCES also suffered greater losses in funding than other statistical 
agencies, while the investment in statistical activity, in 
general, declined by 18 percent between 1980 and 1984, NCES 
experienced a 28-percent reduction. 

Evaluation also was hard-hit. Funds spent on evaluation contracts 
declined by 63 percent; in general, resources for evaluation in 
nondefense federal departments and agencies dropped by 3 7 percent. 
Although all three types of information showed larger reductions 
than similar agencies, the greatest discrepancy was fo- the 
research function, followed by evaluation activities. 

For research and statistical and evaluative information, the 
patterns of declines in funding were consistent and precipitous. 
Tney paralleled the reductions in awards discussed above. Further, 
the consistency of decline in resources across these three types of 
information suggests across-the-board reductions in information 
rather than a substitution of statistics for either recearch or 
evaluation. 
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The Role of Mandates 



The decline in funds interacted with two other factors. First, 
although all information-gathering activities were affected by 
Budget constraints, congressionally mandated activities received 
smaller reductions and thereby consumed an increasing share of 
available resources. About 79 percent of NIE's resources for 
research in 1984 went to legislatively required activities such as 
the Educational Resources Information Center (ERIC) and the 
laboratories and centers, in contrast to 55 percent in 198S. 

This shift is important for three reasons. First, while mandates 
can protect an activity by ensuring a sustained level of support, 
other activities may be affected by insufficient funding or 
staffing or both. Information-gathering activities that did not 
carry a mandate were more vulnerable when faced with fiscal 
constraints. 

Second, specially mandated studies have a large but transient 
effect on the operations of information-producing agencies. 
Depending on a study's size or timing, it can consume a substantial 
amount of a unit's resources, incurring opportunity costs with 
regard to other activities. The problem is exacerbated when the 
units experience losses in staff, as we observed. 
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Third, with regard to quality, mandates alone are not sufficient to 
ensure that high-quality information will be available when it is 
needed. For example, the National Vocational Education Data System 
was mandated in 197 6, and after several years it was disapproved by 
the Office of Management and Budget on the grounds of severe 
technical problems. Here the system was mandated with little 
consultation with the department, no resources were specially 
appropriated, and the time for reporting information back to the 
Congress proved to be unrealistic. 

Changes in Leadership and Priorities 

Changes in agency leadership and priorities also powerfully 
affected the production and quality of information. Each of the 
information units changed in top management positions during the 
1980's. For example, NIB had a total of seven different directors 
from 1980 to 1986, three of the seven serving as acting director. 
At least 16 persons served in the five other top management 
positions, one of which was created in 1984. In six cases, 
individuals served on an acting basis or as special assistants. 
NCES and OPBE showed similar patterns. 

The consequences of management changes can be found in the 
operation and priorities of the information-producing units. For 
example, one identified priority for research at NIB in 19 78 was 
the study of complex learning skills. Many studies have shown that 
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gaps among students are greatest in these skills and that this is 
an area where our school system may be falling short. Despite a 3- 
year effort to develop research proposals in this area— an effort 
that resulted in 3Q proposals being recommended for funding by 
panels of experts—no awards were made, a change in directors had 
meant that this was no longer seen as a priority "area. In sum, the 
cycle for research funding (from identification of a priority 
through the award process to the reporting of results) may take 
many years, but the tenure of the director is typically less than a 
year. This is long enough to stop what had been started but not 
long enough to see initiatives through to completion. 

Turning the Situation Around Is Likely 

to Take Time and Require Data and Monitoring 

The situation we have sketched here is complex, and some of the 
problems— particularly with the statistical information systems- 
are long-standing, I believe it will be neither quick nor easy to 
turn the situation around, involving as it does funds, how 
priorities are set between mandates and discretionary studies, 
leadership and staffing, and other factors. 

This view is not shared by the department. In particular, citing 
the many organizational changes initiated since 1985, the 
department believes that our analyses do not reflect the current 
situation. In fact, its response states that "the department has 
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taken clear and decisive action to address most of the problems 
cited in the report " (emphasis added). In support of this claim, 
the department enumerated how the information portfolio had been 
augmented and the topical areas broadened. 

In my view, it is too early to determine whether the organizational 
and procedural changes that have been initiated will adequately 
address the problems we identified in our report— or new ones that 
the changes might create. For example, we know that the department 
has funded many new centers and minicenters that may be collecting 
new data; however, detailed information was not available for us to 
determine how many awards were directed at remedying the 
educational disadvantages of various populations, for example. 
While it is useful to have additional information on the 
department's recent efforts, assessing whether the information that 
is to be produced by these activities will be available and high 
enough in quality requires more fine-grained evidence. 

For example, the findings of our report could serve as a partial 
baseline against which to assess the effects of departmental 
initiatives to improve the technical adequacy, relevance, and 
timeliness of the particular statistical activities covered by our 
review. More generally, the framework for quality we applied could 
be used as a basis for systematically examining other statistical 
programs. In reviewing the department's comments on our report, 
however, we found no mention of any plans to assess progress 
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empirically. Making plans for a formal evaluation of recent 
changes would be a most timely and useful endeavor. 



For other nonstatistical information-gathering activities, a 
fundamental part of any assessment of progress is the specification 
of the appropriate indicators of success. In the case of research, 
it seems to me that more is needed than a simple list of 
activities. As our review shows, several types of evidence are 
needed to determine whether appropriate information is being 
produced. Phrased as questions, they are, what educational areas 
are being investigated? what target groups is information being 
gathered on? who is responsible for developing the research 
agenda? what type of procurement process is employed? what 
fraction of the awards is devoted to new data collection versus 
support services? Will the new data that are to be produced be 
sufficient to address important questions raised by the large range 
of stakeholders in the educational community? Comprehensive 
descriptive information will provide the needed basis for 
discussing the implications of various decisions that are made. 
Where information gaps are uncovered, repr ioritizat ion or 
augmentation could be initiated. Of course, ensuring the technical 
quality of the research that is funded must remain a high priority 
as well. 
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Summary 



In summary, we have found serious problems affecting the production 
of high-quality information on the condition of education. As I 
said earlier, there is no simple solution to these problems. 
Insufficient funding is directly associated with some of the 
declines in information production. But it is unlikely that merely 
providing more money will allow the department to recover from the 
losses engendered by the reductions in awards. Further, mandating 
particular studies will not work unless resources (staff, time, and 
money) are available. Technical oversight is probably needed to 
ensure that high-quality information is produced. Finally, 
broad-based oversight— that takes into account the diversity of 
information needs of many potential users— concerning priorities on 
what is to be collected, on whom, and under what type of funding 
mechanism seems necessary to avoid many of the notable losses in 
information that we observed. 

Mr. Chairman, this concludes my statement. I will be happy to 
answer any questions the subcommittee may have. 
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